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Two theorems are proved that characterize the matrices used to construct 
systematic error-correcting codes. A lower bound on the number of required 
check bits is derived, and it is shown that, in certain cases, this bound for 
systematic codes is identical with Plotkin's bound on the size of any error- 
correcting code. A linear program whose solutions correspond directly to a 
minimum-redundancy error-correcting code is derived. This linear program 
can be solved by an algorithm that is essentially the simplex method modified 
to produce integer solutions. Explicit solutions in closed form that specify 
the cod-es directly are derived for the cases when the specified code parameters 
satisfy certain restrictions. Several theorems are proved about minimum re- 
dundancy codes with related parameters. 

I. INTRODUCTION 

This paper is concerned with the problem of transmitting binary 
signals over a noisy channel. Some situations in which this problem 
occurs are: when telephone lines are being used to transmit data in 
binary form; when an imperfect medium such as magnetic tape or a 
photographic emulsion is used to store binary data; or when operations 
on binary signals are being carried out by means of circuits constructed 
of devices such as relays, diodes or transistors, which have a probability 
of error. It has been shown by Shannon 1 that it is possible to add redun- 
dant bits to the transmitted messages so as to reduce the probability 
of error in the received messages to an arbitrarily small quantity. Since 
Shannon did not exhibit efficient codes for achieving this reduction in 
error probability, considerable attention has been devoted to the search 
for useful coding schemes. The usefulness of a coding scheme is deter- 
mined by the number of redundant bits that must be added, by the 
complexity of the equipment required for inserting the redundant bits 
before transmission and for removing the redundant bits and correcting 
errors after transmission, and by the error-correcting capabilities. 

1485 
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In 1950, Hamming 2 published schemes for constructing codes for (a) 
detecting the presence of an error in one out of n bits, (b) correcting 
an error in one out of n bits or (c) correcting an error in one out of n 
bits and detecting errors in two out of n bits. In all these codes, it is 
possible to separate the transmitted message into information or message 
bits and redundant or check bits. Hamming defined codes that have 
this property as systematic codes, and proved that all systematic codes 
can be constructed by means of parity constraints on the transmitted 
bits. He also proved that the codes that he constructed contained the 
minimum number of check or redundant bits. While Hamming did not 
obtain any codes for correcting more than one error, he did show that 
a code for correcting e errors can always be changed into a code for 
correcting e errors and detecting e + 1 errors by adding one extra check 
bit that makes the over-all parity of the transmitted message always 
even. 

A procedure for constructing codes for multiple errors was obtained 
by Reed 3 and Muller. 4 The resulting codes are commonly called lleed- 
Muller codes, since they were obtained independently by both Reed 
and Muller. A Reed-Muller code can be constructed for detecting e 
errors whenever e is a power of two (e = 2'). The number of bits in the 
resulting code will also be a power of two. This paper presents a method 
for constructing minimum-redundancy codes for correcting or detecting 
any specified number of errors. 

II. THE HAMMING MATRIX 

A binary word is defined as a sequence of n binary digits, x = x&2 • • • 
x„ ; and the distance between two binary words is denned as d(x, y) = 
(xi ® yi) + (x 2 © y 2 ) + ■ • • + {x n © ijn)* which is equal to the 
number of bit locations in which the two words differ. An e-error-correct- 
ing code 2 is a collection of binary words for which the distance between 
any two words is greater than or equal to 2e + 1 . If an error-correcting 
code consists of all binary words whose digits satisfy certain parity- 
check requirements, the code is called a systematic error-correcting code. 
For example, the collection of six-bit binary words that satisfy x A © 
Xi © x 6 = 0, xi © Xi © .r 6 = and Xi © x% © x 6 = forms a one-error- 
correcting code. The problem of obtaining a systematic error-correcting 
code is equivalent to that of finding a set of parity-check requirements 
that will generate a set of words with the required distance property. 

The parity-check requirements can be specified by a matrix of zeros 
and ones in which the jth column corresponds to the jth bit of the 



The symbol © represents addition modulo two. 
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binary code words and the ith row corresponds to the ith parity check. 
The entry in the ith row and j'th column is one if the jth bit is involved 
in the ith parity check and is zero otherwise. This matrix will be called 
a Hamming matrix, and its elements will be represented by the symbol 
h,j . This is the Hamming matrix for parity rules x 3 ® Xa ® .r 5 = 0, 
x-2 ® Xi ® x e = 0, Xi ® x b ® x G = 0: 



Xi x% x 3 Xi Xb Xg 

"0 1 110" 

10 10 1 

10 11 



= [kil 



The first problem considered in this paper is that of characterizing 
Hamming matrices by determining the necessary and sufficient condi- 
tions that a matrix of zeros and ones be a Hamming matrix for a code 
with minimum distance d (between any two code words). 

In the following, the binary code words will be represented by column 
matrices, 

*~Xx 



.c = 



.'<••> 



and the Boolean 'product of two matrices with elements a,-j and hj will 
be defined as a matrix [c,j] = [a,;] ° [&,-,] with elements c,j = £ a ik b k j 

k 

(modulo 2). 
Exam-pie 1: 
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Definition: A matrix of zeros and ones with k rows and n columns is 
the Hamming matrix for a code of minimum distance d if and only if 
d(x, y) ^ d for all x and y (x ^ y) for which [H\° x ] = 0] and \H\og\ = 
0], where 0] represents a column matrix of k zeros. 

Definition: The weight of a matrix, w[aij], is equal to the number of 
entries of the matrix which are equal to one (for matrices of only zeros 
and ones). 

Lemma / : <V(0, x) = w[x], where represents a sequence of n zeros. 

Definition: The sum (modulo 2) of two or more columns of a matrix 
is the column matrix with each element equal to the sum modulo 2 of 
the elements in the same row of the columns being summed. 
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Lemma 2: If d(x, y) = di , then y] = x ® z], where w[z] = d\ . 

Theorem 1 : H is the Hamming matrix for a code of minimum distance, 
d, if and only if [H]°z] ^ 0] for all z] (2] 9* 0]) for which w[z] < d. 
Proof: First suppose the H is a Hamming matrix for a code of minimum 
distance d and that w[z] < d; then [H]°0] = 0] and d(0, z) < d, so that 
[H]°z] cannot equal 0], by the definition of a Hamming matrix. Next 
suppose that [H]°z] ^ 0] for all z] (z] ^ 0]) for which w[z] < d, and 
d(x, y) < d. Then y] can be expressed as x ® z], where w[z] < d; and 
[H]°y] = [H]°x ®z} = [H]°x] + [H]°z\. Thus, if z is a code word, [H]°x] = 
0], y cannot be a code word, since [H]°y] = [H]ox] + [H]<=z] = 0] + 
[H]°z] ^ 0]. This shows that, if [H]oz] 7^ for all z] with w[z] < d, then 
d(x, y) must be equal to or greater than d for all x and y with [H]°x] = 
and [H]oy] = 0. 

Corollary 1 : H is the Hamming matrix for a code of minimum distance 
d if and only if no set of d — 1 or fewer columns sums to the all-zero 
column. 

Proof: If d — 1 or fewer columns sum to zero there is a corresponding z 
with w[z] < d such that [H\°z] = 0]. 

This theorem makes it possible to attack the problem of finding a 
systematic code with the specified d by constructing a matrix satisfying 
the given conditions. However, no satisfactory procedure for construct- 
ing such a matrix directly is known, and the construction procedure to 
be developed here is based on Theorem 2, which characterizes the parity- 
check matrix, a submatrix of the Hamming matrix. 

III. THE PARITY-CHECK MATRIX 

Hamming showed that a Hamming matrix can always be put in the 
form of a k X k unit matrix (matrix with ones on the main diagonal 
and zeros elsewhere) and a k X n — k arbitrary matrix called the parity- 
check matrix (see Ref. 2, Section 7). This form of the Hamming matrix 
will be called the standard form. In the following it will be assumed that 
the Hamming matrices are always in standard form. 

It is customary to use the term redundant bits or check bits for the 
bits of the code words which correspond to columns of the unit matrix 
part of the Hamming matrix. The remaining n — k = m bits are called 
information or message bits. This usage derives from the fact that each 
of the check bits occurs in only one of the parity checks, and therefore 
the values of each check bit can be calculated directly from the values 
of the information bits, independent of the values of the other check 
bits. If the elements of the parity-check matrix are denoted by pa , 
the check bits (w,) are obtained from the message bits (xj) according 
to the following expression : 
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Ui = 2 Pa Xj 

;=1 



(modulo 2). 



(1) 



A systematic error-correcting code is thus completely specified by the 
parity-check matrix. The main object of this paper is to present methods 
for obtaining parity-check matrices corresponding to systematic codes 
that have a specified minimum distance d between any pair of code 
words and requiring the minimum number of check bits. 

The following is an example of a systematic code of minimum distance 
3 (one-error-correcting code) having two message bits and three check 
bits. 

Example 2: 

[~1 r 

Matrix: P = 1 

1 



equations : 



code: 



Ui = Xi ® Xo , 
U 2 = Xi , 

«3 = xi ; 

U\ W2 U 3 Xi Xi 

0' 
10 10 1 
110 10 
1111 



The method to be used for constructing parity-check matrices depends 
on the following theorem. 

Theorem 2 : P is a parity-check matrix for a code of minimum distance 
d if and only if: 

i. the weight of each column of P is greater than or equal to d — 1 ; 

ii. the weight of the sum (modulo 2) of ./ columns is greater than 
or equal to d — J . 

Proof: First, suppose that the conditions of the theorem are not satisfied, 
and consider the Hamming matrix made up of a unit matrix and the 
given P matrix. If there is a column of P with weight w x < d — 1, then 
the sum of this column and W\ of the unit columns (one unit column for 
each one entry of the column of P) will be equal to zero. Since the total 
number of columns involved in this sum is Wi + 1 < d, the conditions 
of the Corollary I are violated and P cannot correspond to a code of 
minimum distance d. Similarly, if the sum of ,/ columns has weight 
to j < d — ./, these J columns of P plus Wj unit columns will sum to the 
all-zero column. The total number of columns summed is J + Wj < J + 
d — J = d, again violating the conditions of the Corollary 1. Thus, 
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unless conditions i and ii are satisfied by P it cannot be the parity-check 
matrix for a code of minimum distance d. 

Next, suppose that conditions i and ii of this theorem are satisfied, 
and consider which combinations of columns will sum to the all-zero 
column. No combination involving only unit columns can sum to zero, 
since these are linearly independent. As discussed in the preceding 
paragraph, any combination involving only one of the columns of the 
P matrix will contain ivi + 1 ^ d columns, and any combination involv- 
ing more than one of the columns of the P matrix will contain ./ + Wj ^ 
J + d — J = d columns. Thus, any combination of columns that sums 
to the all-zero column must involve at least d columns. The conditions 
of Corollary 1 are satisfied and P corresponds to a code of minimum 
distance (/. 

In this paper the construction of error-correcting codes will be based 
on finding matrices which satisfy the conditions of Theorem 2. The 
matrices will be obtained directly from the solutions to a set of linear 
inequalities. 

IV. FORMATION OF LINEAR PROGRAM 

In order to check that a given matrix P satisfies the conditions of 
Theorem 2, it is necessary to form the sums modulo 2 of all pairs of 
columns of P, compute the weights, and compare the weights with 
d — 2 ; then this must be repeated for all triples of columns, comparing 
with d — 3; all quadruples of columns, comparing with d — 4, etc. A 
systematic procedure for doing this can be given in terms of the following 
definition. 

Definition: Pj ( J = 1, 2, • ■ ■ m) is the matrix formed from P by 
taking, as the columns of Pj , the sums of all possible combinations of ./ 
columns of P (Pi is identical with P). 

Example 3: P is the parity check matrix for a code of minimum dis- 
tance 3 since the weight of each column of P is at least 3 — 1 = 2, and 
the weight of each column of P 2 is at least 3 — 2 = 1 : 
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A method for checking a matrix P is to form P 2 , P-s , 
verify that the weight of each column of Pj is at least d — J. While 
this method is quite satisfactory for verifying that a given matrix 
satisfies the conditions for a parity-check matrix of a code of minimum 
distance d, it is of little use for the more important problem of construct- 
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ing a matrix satisfying these conditions. For this reason, a modified 
method for testing a matrix will be presented as a preliminary to the 
discussion of methods for constructing parity-check matrices. 

In any k X m parity-check matrix there are only 2'" — 1 different 
rows that can occur, since the all-zero row never appears in such a 
matrix. This does not mean that the total number of rows cannot be 
larger than 2'" — 1, since the same row may appear more than once. 
For any given m, it is possible to compute the rows of P> , P 3 , • • •, P„, 
that correspond to each possible row of P. Any specific P matrix with 
in columns can then be tested by selecting the appropriate Pj rows, 
taking into account any multiple occurrences of rows in the P matrix 
being tested. This procedure can be stated more precisely in terms of 
the following definitions. 

Definition: P'" is the matrix having m columns (and 2'" — 1 rows) in 
which each possible wi-bit binary word, except the all-zero word, appears 
exactly once. The rows are ordered in the following fashion : 

First, all the rows containing a single one are written down. These 
rows are ordered so that, when the rows are interpreted as binary 
numbers, they occur in decreasing arithmetic order (this means that 
the first m rows form a unit matrix). Next, the rows containing exactly 
two ones are written down, with these rows arranged so that they occur 
in decreasing arithmetic order. This procedure is repeated by writing 
down the rows with three ones, four ones, etc. until finally the row with 
m ones is written down. Within each set of rows that all contain the 
same number of ones, the rows are arranged in decreasing arithmetic order. 

Example 4 : 
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By making use of this definition of P'", a concise specification of any P 
matrix having m columns can be given by listing which rows of P m 
occur in P. 

Definition: If P is a parity-check matrix having m columns, then 
Zi(P), i = 1, 2, • • • 2"' — 1, is equal to the number of times that the 
ith row of P m occurs in P. Usually z,(P) will be written simply as z { 
when the appropriate P is clear from the context. 

Example 5: 

2l(P) = 1 
2 2 (P) = 2 

P =111, z 3 (P) = . 

*(P) = o 

*(P) = 1 

*(P) = o 

zi(P) = 1 

It is now possible to state the requirements for parity-check matrices 
in terms of Zi(P), P m and Pj m , where P/' is the matrix formed of all 
sums of J columns of P m . 

Theorem 8: A matrix P with each entry equal to zero or one is a 
parity-check matrix for a code of minimum distance d if and only if 
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h(P) 2 2 (P) • • • ft--i(P)] [P/1 ^ [d - J, d - j, ■ ■ -, d - jy 



(2) 



for I ^ J ^* m. 
Proof: For J = 1, 



h(P) 2 2 (P) • • • «i--i(P)] [Pi"l 



is just equal to the weights of the columns of P, since each row of P m 
is multiplied by the number of times it occurs in P [z,(P)] and then a 
sum for each column is formed. Similarly, for J ^ 1, 

[z,(P) Sa(P) • • • *2»-i(P)] [P/ M ] 

is equal to the weights of the columns of Pj . By Theorem 1, these 
weights must be greater than or equal to d — J. 



* The multiplication here is ordinary matrix multiplication. The inequality is 
satisfied if, and only if, each element of the row matrix obtained by the multiplica- 
tion is at least as large as the corresponding element of the row matrix given on the 
right side of the inequality. 
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Example 6: 
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\ZlZ2---Z7]-\P*) = [0001111] 



Wi--- zi\-\P^\ = [0001111] 
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[2l22---Z7HP 3 3 ] = [0001111] 



Zx = 

zi = 

23 = 0' 

2 4 = 1 

2 5 = 1 

26 = 1 

2 7 = 1 



= [333]^[d-l,d- \,d- I J, 



= [222] = [d-2,d-2,d-2], 



= [1] ^ [d - 3]. 



Thus, P is the parity-check matrix for a code of minimum distance 4. 

Theorem 3 is merely a restatement of Theorem 2 using different 
notation. The reason for introducing this new notation is that, by means 
of Theorem 3, the problem of constructing minimum redundancy codes 
can be formulated as an integer linear programming problem. 

Lemma 8: 



*=E 



Zi 
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Proof: By definition, k equals the number of rows in the matrix P and 
zt equals the number of times that row * of P m occurs in P. Since each 
row of P is identical with some row of P'", the total number of rows of 
P just equals 

2 m -l 

£ **• 

i=I 

Theorem 4: The problem of finding a minimum-redundancy, systematic, 
error-collecting code for specified values of m (the number of message 
bits) and d (the minimum distance between any pair of code words) 
is equivalent to the problem of solving the following linear program: 

minimize: 

2 m -l 

k = £ * 

subject to : 

(LP) 

( 1 ) Zi integers, 

(2) Zi ^ 0, 

(3) [«, % ■ • ■ ft^J • [Pj m ] ^ [d - J, d- ./,•••, f/ - J] 

for J = 1, 2, • • • m. 

Proof: The solution to (LP) will be a set of values for z x , z 2 , • • •, «2»-i • 
These values can be used to construct a matrix P by interpreting them 
as Zi(P). By Theorem 3, these values for z,(P) must satisfy (LP-3) 
and by the definition of z t (P) they must satisfy (LP-1, 2). Since a 
minimum-redundancy code is desired, it is necessary to minimize k. 
Lemma 3 establishes the expression for k in terms of z t {P). 

The remainder of this paper will consist mainly of obtaining solutions 
to this linear program. 

V. BOUNDS ON REDUNDANCY 

In a certain sense, the formulation of (LP) solves the problem of 
constructing the desired codes, since a numerical procedure exists for 
solving this type of integer linear program." Practically, this procedure 
is of limited usefulness, since the size of the program to be solved soon 
exceeds the capability of the largest electronic computer. Also, numerical 
solutions do not provide information about the interrelations among 
various codes with different parameters. A much more desirable solu- 
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tion would be a closed solution of (LP) in which the values of Z\ , 2 2 , ■ ■ ■, 
z 2 '»-i and k are expressed as functions of m and d. The derivation of such 
closed solutions and of various properties relating solutions for different 
parameters will constitute the remainder of this paper. 

The first step in obtaining solutions of (LP) will be to remove the 
matrix notation and express (LP-3) as a set of simultaneous inequalities. 
This is done to simplify the proofs of the theorems to follow. 

The inequalities represented by (LP-3) can be expressed in terms of 
a single matrix by defining a matrix A'" in which all of the columns of 
P\", P-'", ■■-, P m '" appear. 6 

Definition: The matrix A'" is formed as follows: 

(1) The first m columns of A"' are identical with P'". 

(2) The jth column of A'" (j > m) is formed by taking the sum 
modulo 2 of the columns of P'" that have one entries in thej'th row of P"\ 
When the value of m is clear from the context, A'" will be written as A. 

Example 7: For m — 3, 
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A 3 = 



"10 110 1 
10 10 11 
10 

1 
1 

11110 
1110 1 



1 1 1 

10 110 
110 10 



The fifth column of A' is formed from the sum modulo 2 of the first 
and third columns of P 3 since the fifth row of P has ones in the first 
and third columns, etc. 

Definition: Let €j(m) be defined as follows: 



tj(m) = 1 
ej(m) = 2 

€j{m) = s 



for 1 ^ j ^ m, 

for 1 + m ^ j ^ m +(^j > 

for EhsiSEM, 



tj(m) = m for 2-i 



m 



^ j ^ T - 1. 



Theorem 3 can be stated in terms of .4'" as follows: 

Theorem 3': A matrix P is a parity-check matrix for a code of mini- 
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mum distance d if and only if: 

[ Zl (P) z»(P) • • • 2.»_i(P)] • [A n ] ^[w lWi --- w 2m -i], (2') 

where wj = d — tj(m). 

Some properties of A'" are stated in the following lemmas, in which 
a,/" represents the element of the ith row and jth. column of A m . When 
there is no ambiguity possible, a,/" is written simply as a {j . Proofs 
will be given in the Appendix. 

Lemma 4: The matrix A"' can be partitioned into submatrices as 
follows: 

a'" = [/y : p 2 '" j /Y" j ••• j P m '"l 

Lemma 5: 

an — aattji ® anfLji ® • ■ ■ ® ai m aj m for j > m. 

Lemma 6: The transpose of A is identical with A : A T = A, or an = 

an . 

Lemma 7: 

2"'-l 2">-l 

V* m V^ „ m a"<— 1 

2- Oii = 2^ «U = ^ • 

i=l i=l 

Lemma 8: 

m 

£ a.7 m = «i(m). 

Lemma 9: The inverse of A, A -1 , is obtained from A by replacing 
each one entry of A by 2 1- '" and replacing each zero entry by -2 1_ffl : 

atf 1 = 2 1- '" if an =1, and 

fl« _1 = -2 1 "'" if «.; = 0. 

Theorem 3 can be stated directly in terms of the a,-/" as follows: 
Theorem 3" ': A matrix P is a parity-check matrix for a code of mini- 
mum distance d if and only if: 

2">-l 

X) a-ijZi ^ Wj, (2") 

where Wj = d — ej(m). 

The corresponding formulation for the program (LP) is 

minimize : 

2 m -l 

k = £ *i 
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subject to: 

(LP') 

( 1 ) Zi integers, 

(2) Zi ^ 0, 

2">-1 

(3) £ anZi ^ wj , 

where Wj = d — ej(m). 

The following theorem presents a lower bound on k, the number of 
check bits that are required for specified values of m, the number of 
information bits and d, the minimum distance between code words. 

Theorem 5: For an error-correcting code having minimum distance d 
and m message bits, the number of check bits, k, must satisfy 

k ^ l—.,—,- 1 )d - m. (3) 



— \ Om-1 



Proof: By (2"), 


















2 m -l 

£ a, 

i=l 


IjZi 


> 


w j} 






2 m -l 

E 


! E a. 

1=1 


jZi 


> 


2 m -l 

E 


Wj, 




2-n-l 

E 

i=i 


2 m -l 

E«, 


IjZi 


> 


2 m -l 
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IV j, 



2 m -l 2"»— 1 2 m -l 

2 z» 2 «.y ^ H Wj. 
i=i y=i i=i 



But, by Lemma 7, 



2"'-l 



an = 2 , 

SO 

i=i i=i 

and 

2 m -l 2 m -l 

fc = 2 z* ^ 2 1_m 2 W/i by Lemma 3. (») 

i=i j=i 

By the definition oi Wj , 
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but 



§(:) 



s = m2 m 



so 



E u>i = (2 m - 1)<*-m2""\ 

Substituting this in («) yields 

A; ^ 2 1 ~ m [(2 m - 1) d - m2 m " 1 ] 

or 

A- > ( - - ~ - ) d - m . 






Since the total number of bits in each code word, n, is just equal to 
m + />•, this bound on k yields a bound on n. 

Corollary 2: For an error-correcting code having minimum distance 
(/ and m message bits, the total number of bits in each code word must 
satisfy: 



n =£ 



(fe-> 



a (^=^ U. W 



If d < m, the bounds given in (3) and (-4) can be improved, since 
some of the wj in (2") will be negative and should be replaced by zeros. 

Corollary 3: When d < m, k and n must satisfy the following inequali- 
ties: 

**(^) < — + *-£(:)<—>. < 3 '> 

»*(tV*~ £(:)<•-* (4,) 

Proof: From (ii) of Theorem 5, 

2 m -l m / \ 

E wy = El™) (<* - s), 

3=1 «=1 \ S / 

but when to,- < it can be replaced by 0. This is equivalent to defining 
w/ = d — ej(m) ford ^ «>(»») and w/ = for d < tj(m). Then, 

2"'— 1 d / \ 

E W — E ( ) (d — s) for d < m, 



8 = 1 \S 
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or 



E W = E( ) (<* - -0 + £ ('") (« - <D ford < m 



w 



= (2"'- l)d-m2 ro - 1 + £ (" fr )( S -d). 



■/??. 



By («) of Theorem 5, 

2"- 1 



> 9 



3=1 



> V 



(2 m - l)d- w2"- 1 + U (" v )(s-d) 

8 =,1+1 \S 



t)l 



m 



fcf^) *-*+*- jut; <*-«>• 

Whenever d ^ fe2"' _1 , the bounds (3) and (4) are not integers and 
therefore cannot be met exactly. 

Definition: Let {N\ equal N if N is an integer and equal the smallest 
integer larger than N if N is not an integer. 

Definition: Let 



and 



k*(m,d) = 



n*(m,d) = 



\ 2"- 1 / 



d —m 



2 m - 1 

Om-l 



d . 



.Since the total number of bits per code word and the number of check 
bits must both be integers, the following inequalities follow directly 
from Theorem 5 and Corollary 2. 

Corollary 4: For an error-correcting code of minimum distance d and 
having m message bits: 



k > k* 



and 



n ^ n* 
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VI. MINIMUM-REDUNDANCY CODES 

Since n and k must be integers, the bounds given by (2) and (3) can 
be met exactly only when d is divisible by 2 W ~\ that is, when d can be 
written as d = h2"'~\ where h is a positive integer. The following theorem 
shows that the bounds can always be achieved in these cases. The ap- 
propriate P matrix is formed by including each possible distinct row h 
times, except for rows of weight one, which are included only h — 1 
times. 

Theorem 6: Whenever d = h2'"~ l , where h is any positive integer, a 
minimum-redundancy code exists with 

k = h(2 m - 1) - to, 

n = h(2'" - 1) 

and 

Zi = h - 1 for 1 ^ i ^ to 

= h for m + 1 ^ i ^ 2 m - 1. 

Proof: Let Zi = h — 1 for 1 ^ i ^ to and z, = h for to + 1 ^ i S 
2 m - 1. Then, 

2">— 1 2 m — 1 m 

23 flu^i = 23 Jmz« — 23 o« > 

,=1 i=l i=l 

2 m — 1 2 m — 1 

2 A«« = A 13 a «i = A2 m_1 by Lemma 7 , 



Thus, 



23 a u = *i( m ) by Lemma 8. 



E a, y2i = A2"- 1 - € y(m) . 



2 m — 1 

E 

1=1 



But, by Theorem 3", this is exactly the condition for a code of minimum 
distance h2 m ~ 1 . 

The number of check bits, k, is given by 

2 m -l 2">— 1 m 

k= 23 *- E A-S(D 

i=i i=i i=i 

= A(2 ra - 1) - m. 
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By Theorem 5, 



* ^ ( ; W) {hr ~ l 



) — m 



^ (2 m - 1) h - m. 

Therefore, the code is a minimum-redundancy code. 
Example 8: For m = 3, d = 8 = /t-2 m_1 = 2-2 3 " 1 : 

k = h(2 m - 1) - m = 2(2 3 - 1) - 3 = 11, 



P = 



[1 ol 


1 


1 


1 1 


1 1 


1 1 


1 1 


1 1 


1 1 


1 1 1 


.1 1 1_ 



and 



Zl = 22 = Z 3 = 1, 



Zi = Zf, = Z 6 = Zt = 2. 

Theorem 5 can be extended to the case when d = h2 m ~ 1 — 1 by 
means of the following theorem, which has originally proved by Ham- 
ming. 

Theorem 7 {Hamming): From any minimum-redundancy code* con- 
taining n bits per code word and having minimum distance d, with d an 
even number, it is possible to obtain a minimum-redundancy code con- 
taining n — 1 bits per code word and having minimum distance d — 1 
by removing one of the bits from each of the code words (the same bit 
must be removed from each word). If the original code was a systematic 
code, the bit removed should be one of the check bits. 

Conversely, from any minimum-redundancy codef containing n bits 
per code word and having minimum distance d, with d an odd number, 
it is possible to obtain a minimum-redundancy code with n + 1 bits 
per code word and having minimum distance d + 1. This is done by 



Not necessarily a systematic code. 
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adding a check bit that is a parity check over all of the bits of each code 
word. 

Corollary 5: If Z\ , 22 , • ■ ■ , z 2 m_i specify a minimum-redundancy syste- 
matic code for (/, where d is an even number, it is possible to obtain a 
minimum-redundancy code for d — 1 by decreasing any nonzero 2, by 
one. 

Corollary 6: Whenever d = h2 m ~ l — 1, where h is any positive inte- 
ger, a minimum-redundancy code exists with 

k = h(2'" - 1) - (m - 1), 

n = h(V' - 1) - 1. 

This follows directly from Theorem 7 and Corollary 5. 
There is a large class of codes for which 



( 2 m - l \ 

y 2>n-l J 



d 



is not an integer, but for which minimum-redundancy codes with k = k* 
can be derived. 

Theorem 8: Whenever d = hi 2 m_1 — 2 hi , where hi is a positive integer 
and h 2 is a positive integer with h 2 < m — 1, there exists a minimum- 
redundancy code with 

k = h(2 m - 1) - 2" 2+1 - m + 1, 



and 



where 



and 



n = h(2 m - 1) - 2" 2+1 + 1 



Z% — Zi Z{ , 

■ = hi — 1 for 1 ^ i ^ m 

= K for m+Uig2 m - 1, 



z" = 1 if the corresponding row of .4 '" has all zeros in its 
first m — h 2 — 1 columns 

= if the corresponding row of A m does not have 
all zeros in its first m — h 2 — 1 columns. 

Proof: Let Zi , z' and z/' be defined as in the statement of the theorem. 



ERROR-CORRECTING CODES 1503 

By the methods used in proving Theorem C, it follows that 

E diiZi' = hr- 1 - ej(m). 
1=1 

Consider 

2'"-l 

E a ijZ /'. 

i=i 

This is equal to 



2*2+1-1 

1=1 

where the bu are the entries of a matrix B, which is made up of those 
rows of A"' for which the corresponding z" are equal to one. The first m — 
fa — 1 columns of B contain all zeros. Therefore, 

2*2+1—1 

E hi = for 1 £j £m- ht- 1. 

1=1 

The next fa + 1 columns of B are identical with P* 8+1 , since each com- 
bination of zeros and ones (except the all-zero combination) occurs 
exactly once. Thus, since the weight of each column of P' 1 is 2' 2 , 

2*2+1-1 

E bu = 2 hn - for m - fa ^ j ^ m. 
i=i 

Every other column of B is formed from the sum modulo 2 of some of 
the first m columns. Since the all-zero columns do not have any effect 
on the sum modulo 2 operation, every other column of B is equal either 
to one of the columns for m — fa 5S j ^ m or to the sum modulo 2 of 
several of these columns. Thus, every remaining column of B is identi- 
cal with some column of A hi+1 . Therefore, for m + 1 ^ j ^ 2"' — 1, 

2*2+1—1 2*2+1 — 1 

E, V^ „ h % +l Q ft 2 



Thus, 



2 »._1 2*2+1-1 

E a ijZi " = E 6.7 = for 1 ^j ^ m - fa- 1 

i=l i=l 



= 2" 2 for m - fa ^ j £ 2 m - 1 , 



and 



2"'-l 

E 

1=1 



E a»i2i = fa2 m ~ l - ey(m) - for 1 £ j g m - fa - 1 
= fa2 m ~ x - ej(m) - 2 k * for m - ft 2 £ j £ 2'" - 1 . 
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Thus, 

E *& ^ M""" 1 - 2* 2 - €j(m) , 

1=1 

or the given z t - satisfy the requirements for a code with d = hi 2 m ~ 1 — 2 2 . 
This proves that a code constructed from the given z,- will have d = 
h\2 m ~ l — 2* 2 . A proof must now be given for the fact that the result- 
ing code is minimum-redundancy 

om—i 2 m — 1 2 m -l 

fc = ^2 zt = X! z/ — ^2 z" . 

i=i 1=1 t=i 

The z{ are the same as the z t of Theorem 6 ; therefore, 



£z/ = h{2 m - 1) - m. 
i=i 

Since there are 2 Az+1 — 1 rows of A m that have all zeros in the first 
m — hi — 1 columns, 

2 vV = 2* 2+1 - 1 
i—i 

and 

k = hi(2 m - 1) - m - 2" 2+1 + 1. 
Now, 

fc* (m,^i2 m - 1 - 2* 2 ) = {(^r 1 ) (W -1 ~ 2" 2 ) - m 

which can be rewritten as 

k* = {(2'" - l)fo - 2* 2+1 + 2 1+ " 2 "" - m), 
but, since m > 1 + hi , 

2 l+A 2 -m < ^ 

so that 

fc* = (2 m - \)hy - 2 h2+1 - m + 1, 

and therefore /c = /c* and the code is minimum-redundancy. 

Corollary 7: Whenever d = /&i2 m_1 — 2'' 2 — 1, where hi is a positive 
integer and h 2 is a positive integer with h 2 < m — 1, a minimum- 
redundancy code can be obtained from the code of Theorem 8 by the 
method given in Corollary 5. 

Minimum-redundancy codes for d = 2, 3 and 4 were given in a paper 
by Hamming. A code for d = 2 can be obtained by using all w-bit words 
that contain an even number of ones, since Theorem 2 just requires each 
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column of P to contain at least one one. Thus, the minimum-redundancy 
codes for d = 2 all have k = 1. 

For codes of distance 3, Theorem 2 requires that each column of P 
contain at least two ones and that no two columns be identical. In this 
case, m is equal to the number of different columns having k rows, and 
at least two entries equal to 1 ; or m = 2 — k — 1 . 

VII. RELATIONSHIPS AMONG CODES 

For values of d which are greater than 4 and do not satisfy the condi- 
tions of either Theorem G or Theorem 8, it has not been possible to 
obtain closed solutions of the linear program (LP). Computation using 
Gomory's algorithm 5 is necessary to obtain minimum-redundancy 
codes for these values of d. The following theorems present various 
general properties of minimum-redundancy codes that are useful in 
obtaining codes for new values of d from the codes obtained by use of 
the algorithm. 

Definition: Let K(m,d) be the minimum value of fc that is possible 
for a code having m message bits and minimum distance d. 

Definition: Let N{m,d) be the minimum value of n that is possible 
for a code having m message bits and minimum distance d. 

Lemma 10: 

K(m - l,d) ^ K(?ti,d). 

Proof: A parity-check matrix for m — 1 can be obtained from the ma- 
trix for m by simply removing one column. Since the conditions of 
Theorem 2 must be satisfied by the reduced matrix if they were satis- 
fied by the original matrix, the reduced matrix corresponds to a code of 
distance d if the original matrix corresponded to a code of distance d. 
Theorem 9: For m £ d < 2'"" 2 , 

N(m,d) > n*(m,d), 

K(m,d) > k*(m,d). 



Proof: 



k*(m,d) = \ ( " 9w _ 1 ) d — m\ for m ^ d 



= [(2 - 2 l ~ m ) d - m\ 



^ " ~3 " m 



k*(m,d) = 2d - m form ^ d < 2 m \ 

k*(m - M) = 2d - m + 1 form. - 1 ^ d < 2 m ~ 2 
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but 

K(m,d) ^ K(m - l,d), 

K(m - l,d) ^ k*(m - l,d), 

k*(m - l,d) > k*(m,d) form ^ d < 2"'~ 2 , 

K(m,d) ^ K(m - l,d) ^ k*(m - l,d) > k*(m,d), 

so that 

K(m,d) > k*(m,d) for m ^ d < 2 m ~\ 

Since n = k + m, it follows that 

N(m,d) > n*(m,d) for m ^ d < 2 M " 2 . 

Theorem 10: 

JV(m//, + r/ 2 ) ^ N{m4x) + N(m,d->), 

K(m,di + d 2 ) ^ A'(w//,) + K{m,d->) + »?. 

Proof: Let 2' = [zi' 22' ■ • ■ Z2«-i'] he the values of z> corresponding to 
N(m,di) and 2" = [21" 2 2 " • • • Z2«»-i"] he the values of 2, corresponding 
to N(m,d 2 ). Thus, 



2"'-l 

£ 

1-1 



£ * i" «/ ^ ^1 - «i(*») 



and 



Let 



and 



Then, 



1=1 



2,- - 2,-' + 2," + 1 for 1 ^ i ^ m 

|. = z > + 2 / f or m + i ^ 7 : g 2'" - 1. 



om_l 2""-l 2"'-l "i 

,=1 r=i 1=1 '=1 

^ di - €j(m) + d-. - ey(m) + e/(m) 
^ di + da - ej(m). 
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Thus, Zi satisfy the conditions for a code of distance d\ + d 2 . Further- 
more, 

owi-l 2'n-l 2"'-l m 

22 zi = E «' + E v + £ (i), 

«=1 i"=l i=l i=l 

2"' — 1 

/r(???,rf,+f/ 2 ) = £ z t - = K(m,di) + Kim/h) + m, 
i=i 

7C(m,d, + d») ^ KimA) + K(m,d 2 ) + m. 



so that 



Corollary 8: 

N(m,di + /'2'"" 1 ) ^ iV(»Mi) + fc(2 w - 1), 
K(m,di + /t2'" _l ) ^ K(m,di) + /i(2'" - 1). 

Definition: Let max(zi , 2 2 , ■ ■ • , z m ) equal the largest of the values of 
the Zi . 

Theorem 1 1 : Let Z\ , 2 2 , • • • , z-i»>-\ correspond to a code for which k = 
K(m,d). Then, if max(zi , • • ■ ,z m ) = M, 

K(m - 1//) ^ K(m,d) - M. 

Proof. Let 2/ , (1 :£ i" ^ m) be one of the 2, such that z r = M. Then, 
if the /th column is removed from the matrix P specified by (zi , z» , • ■ •, 
2 2 m-i), the resulting matrix must still correspond to a code of minimum 
distance d (Lemma 10). However, M of the rows of the reduced matrix 
consist of all zeros, since there are J\I rows which contain a one only in 
column /. Thus, these M rows can be removed without affecting the 
minimum distance d. Removal of M rows decreases 1: by M, giving 

K(m - l,d) ^ K(m,d) - M. 

VIII. COMPARISON WITH PLOTKIN's BOUND 

The approach of this paper has been to search for codes which require 
the minimum number of check bits, k, for specified values of m and (/. 
Another common approach to the study of error-correcting codes is to 
specify the total number of bits per code word, n, and the minimum 
distance, d, and then to try to construct codes which contain the largest 
number of messages. A bound on this maximum number of messages 
has been proved by Plotkin. 7 

Theorem 12 (Plotkin): Let A(n,d) equal the maximum number of 
binary ra-bit words in an error-correcting code (not necessarily a syste- 
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matic code), for which the distance between any two code words is at 
least equal to d. Then 

A(n,d) ^ 2^1i f ° r 2d > U ' 

For a systematic code, the number of messages must be a power of 
two, and therefore this bound can be met exactly only when 2d I {2d - n) 
is a power of two. The following theorem shows that, whenever this is 
true, a systematic code exists which does meet the bound. 

Theorem 13: For values of n and d such that 2d/(2d - n) = 2 m , for 
some m, a systematic code exists with m message bits and therefore 2 m 
code words. For such values of n and d, no code of any type is possible 
with more code words. 
Proof: The equation 2d/(2d - n) = 2'" can be written as 



d = 



\2 m - 1/ 



Since d and 2 m_1 are integers, and 2 m — 1 does not divide 2 m_1 , n/(2 m — 1) 
must be an integer. Leth = n/(2 m - 1), then d = h2 m ~\ By Theorem 6, 
a code exists with d = h2'"~ and 

n = »(8- - 1) = (^j) (2" - 1) = . 

and m message bits. 

By Plotkin's theorem, no code with more code words is possible. 

IX. ACKNOWLEDGMENT 

The author wishes to acknowledge the many helpful comments con- 
tributed by D. H. Evans, S. H. Unger and M. C. Paull. Thanks are 
due also to W. Ulrich and C. Y. Lee for many stimulating discussions 
on the general subject of error-correcting codes. 

APPENDIX 

Various proofs have been omitted in order not to disturb the con- 
tinuity of the paper. These proofs will be presented here. 

Lemma 4: The matrix A m can be partitioned into sub matrices as 
follows: 

A m = [pr ; p 2 m \ p 3 m i ••■ j p m m \. 

Proof: By definition, the first m columns of A m are identical with Pi"'. 
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The rows of P'" are ordered so that all rows with one one come first, 
then all rows with two ones, etc. The columns of P'" which must be 
summed to form the jth column of .-1 '" are determined by the jth row 
of P'". Because of the ordering of the rows of P m , all sums of pairs of 
columns of P'" will occur as columns of A m , then all sums of three col- 
umns of P'", etc. Since P/' contains all sums of J columns of P m , A m 
can be partitioned as shown. 

Lemma 5: an = a,ia>i © 0,2^/2 © • • ■ © a{ m aj m , forj > m. 
Proof: Thejth column of A m (j > m) is formed by taking the sum mod- 
ulo 2 of the first m columns of A m which have a one entry in the jth 
row of A m . Thus, if the jth column of A m is denoted by A /", 

Aj m = ajiA™ © ajzA 2 m © • • • © a jm A m m forj > m, 

since the column Ai" is to enter into the sum only if an = 1, etc. It 
follows from this that the ith element of the jth column (j > m) is 
given by 

da = anan © a^a^ © • • • © a im ay„, . 

Lemma 6: 

a<j = an . 

Proof: It follows directly from Lemma 5 that a,-,- = a,-,- for j > m or 
i > m. For j < m and i < m, the definition of A requires that an = 
unless i = j, so that a,-,- = an = for i j* j and, for i = j, an is identi- 
cal with a,i . 
Lemma 7: 

2">-l 2 m -l 

Q-ij = 2-j an = 2 

Proof: The first m columns of A m contain each m-bit binary number, 
except the all-zero number, exactly once. Consider these rows which 
have a one entry in the first column. There must be 2 m_1 such rows, 
since there are 2" ,_1 different (m — l)-bit binary numbers, and each of 
these must occur once in the remaining m — 1 columns. Thus, 

2 m -l 

£ eta" = 2-\ 

i=l 

A similar argument shows that 

2 m_i 

S «* " - 2 m_1 for lg;g m. 
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For j > m, each entry a,/" is the sum modulo 2 of the entries in s of the 
first m columns. Consider these columns. Each s-bit binary number 
must occur exactly 2 m ~* times (except the all-zero number, which oc- 
curs 2"'~* — 1 times), since there are exactly 2"'~* ways to choose the 
entries in the remaining m — s columns. There are 2" different s-bit 
numbers and 2 s-1 of these contain an odd number of one entries. Thus, 
there are (2"'~") (2 S_1 ) = 2"' _1 rows containing an odd number of one 
entries, and hence 2" ,_1 of the a,/" are equal to one. Thus, 

2 m -l 

2 o*i = 2~ _l for j > m - 

Since «,•> = a 7 -,- , it follows that 

E 1 a 1 ; = 2'"- 1 . 
i=i 

Lemma 8: 

m 

Proof: The definition of tj(m) is: 

e,(m) = 1 for 1 ^ j ^ m, 

(til \ 
2/ ' 



€j(m) = s 



for £ ( m ) ^ s £ (™) . 



Consider the first w columns of A 7 ". The first m rows contain a single 
one, since they are all the ra-bit numbers containing one one. The next 

If rows contain two ones, the next (J rows contain three ones, etc. 

Thus, 

S flu™ = «.-(»»)■ 
Since o»-,- = a;, , it follows from this that 

Ea,/"= e,(m). 
i=i 

Lemma 9: If the elements of A -1 are represented by Oiy" , then 

Oif 1 = 2 1 "'" if O.y = 1 
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and 

o l7 _1 = -2 1 "'" if an = 0. 

Proof: The a,, -1 given above are the elements of the inverse of A if and 
only if 

2 IB— i 

b,j = 2 0.-.0,/" 1 = 1 if t = j 



and 








2'"— 1 

8=1 


if » ^ y. 


If i = j, then 








2'"-l 2"'-l 

8 = 1 8=1 


2 m -l 

8 = 1 


which equals 


2 "1-1 

2 1 -' Ea l{ . 

8 = 1 





But 



so that 



If t * ./, 



2"'-l 

/ , a s , = 2" ,_ by Lemma 7, 

8 = 1 



2»'-l 



Z-i Quasi = 1. 



&,-,■ = X) a / Ay ' = £ fliAi . 



2'"-l 

c 

=1 



Three cases will be considered: 

Case 1: i < m and,/ < m. 

In this case, bij = 2 1- '" (number of 11 entries in columns * and j) 
minus 2 1_m (number of 10 entries in columns i and j). By the argument 
used in proving Lemma 7, there are 2"'~ 2 11 entries and 2"' -2 10 entries, 
so that b,,- = (2'"'") (2 m_z ) - (2 1 -'") (2 m - 2 ) = 0. 

Case 2: i < m and./' > m. 

In this case, the elements a s j are formed as the sum modulo 2 of the 
entries in v of the first m columns of A, so that 6,-y = 2 1- '" No — 2 
N e , where 
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No = the number of rows of A which have a 1 in the ith column and 
an odd number of l's in the v columns from which a aj - is formed. 

N e = The number of rows of A which have a 1 in the ith column and 
an even number of l's in the v columns from which a sj is formed. 
Again, by the argument of Lemma 7, in the v + 1 columns consisting 
of column i and the v columns used to form a ti , each different binary 
number (except the all-zero number) must occur exactly 2 m_ * times 
so that N e = iVo and &,-,- = 0. 

Cased: 

A similar argument shows that foj = for the case when 1 > m and 
j > in. 
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